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■Meaningful  human  factors  applications  to  the  design  of  human/computer  tasks  require 
a  quantitative  data  base  that  describes  operator  behavior  as  a  function  of  various 
independent  variables.  Three  classes  of  metrics-operator  satisfaction  ratings,  work 
sampling  procedures,  and  embedded  performance  measurement— are  described  as  impor¬ 
tant  measures  in  evaluating  human/computer  interfaces.  Polynomial  regression 
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ocedures  were  used  to  generate  functional  relationships  between  each  of  these  classes 
of  metrics  and  four  independent  variables  representing  timing  attributes  of  an  interactiv< 
computer  system  used  to  enter  and  update  personnel  records  (system  delay,  display  rate 
keyboard  echo  rate,  and  rollover  buffer  length  of  the  keyboard).  Each  of  the  22  dependen 
variables  in  the  three  classes  of  metrics  showed  different  functional  relationships  amonf 
the  four  system  variables,  but  overall  system  delay  and  keyboard  echo  rate  were  tin 
major  predictors  of  operator  behavior.  Additionally,  the  three  classes  of  metrics  wer« 
combined  into  three  underlying  interface  dimensions  relating  to  operator  production 
waiting,  and  planning  activities. 
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FOREWORD 

This  research  was  performed  under  contract  to  the  Virginia  Polytechnic  Institute  and 
State  University  in  support  of  program  element  63707  (Manpower  Control  System 
Development),  project  Z1170-PN  (Human  Processing  of  Large  Automated  Data  Bases), 
subproject  Z1170-PN.03  (Improving  the  Accuracy  and  Usability  of  Automated  Personnel 
Information  Systems).  It  was  sponsored  by  the  Deputy  Chief  of  Naval  Operations 
(Manpower,  Personnel,  and  Training)  (OP-01). 

The  subproject  was  directed  toward  resolving  fundamental  human  engineering  design 
issues  in  systems  that  contain  man/computer  interfaces.  Preliminary  research  for  this 
subproject,  which  was  performed  under  program  element  62763  (Personnel  and  Training 
Technology),  work  unit  ZF  55-52 1-00 1-002-0 3.03  (Forecasting  New  Task  Requirements) 
resulted  in  an  annotated  bibliography  of  human/computer  transaction  tasks  (NPRDC  TN 
82-14). 

This  report  provides  quantitative  data  on  the  effects  on  operator  performance  of  four 
characteristics  of  computer  systems:  (1)  systems  delay,  (2)  display  rate,  (3)  keyboard  echo 
rate,  and  (4)  rollover  buffer  length  of  the  keyboard.  The  research  was  completed  in  March 
1981  and  results  used  at  NAVPERSRANDCEN  in  research  to  evaluate  human/computer 
interfaces  in  automatic  data  processing  systems.  The  report  is  being  published  at  this 
time  to  make  it  available  to  the  research  community  and  to  others  developing  systems 
requiring  man/computer  interfaces. 

The  contracting  officer's  technical  representatives  were  Mr.  Richard  W.  Obermayer 
and  Mr.  John  S.  Malone. 


JAMES  F.  KELLY,  JR 
Commanding  Officer 


JAMES  W.  TWEEDDALE 
Technical  Director 
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SUMMARY 


Problem 


Navy  personnel  records  are  becoming  more  computerized  and  design  of  the 
human/computer  interface  must  be  considered  to  increase  data  entry  productivity  as  well 
as  reduce  entry  errors.  To  enhance  these  design  considerations,  functional  relationships 
of  operator  performance  are  needed  that  incorporate  a  variety  of  system,  task,  operator, 
and  environmental  factors. 

Objective 

The  purpose  of  this  study  was  to  demonstrate  the  utility  of  using  three  classes  of 
metrics-work  sampling,  embedded  performance  measures,  and  satisfaction  ratings— to 
evaluate  the  human/computer  interface  for  data  entry  of  personnel  records.  Each  class  of 
measures  was  used  to  generate  functional  relationships  between  operator  performance  and 
four  system  parameters— system  response  time,  display  rate,  keyboard  echo  rate,  and 
keyboard  buffer  length. 

Method 

A  simulated  data  entry  task  was  structured  around  a  Navy  personnel  record  task  in 
which  the  operator  was  required  to  use  an  interactive  computer  terminal  to  perform 
either  ADD  or  CHANGE  transactions  on  simulated  pay  order  records.  An  orthogonal, 
central-composite  design  was  used  to  specify  the  data  collection  requirements  for 
evaluating  the  four  system-timing  variables.  A  total  of  400  transactions  was  evaluated 
across  22  different  dependent  variables,  representing  the  three  classes  of  metrics 
evaluated  in  this  study. 

Results 

Both  univariate  and  multivariate  analyses  were  conducted  on  the  data  to  generate  a 
series  of  second-order  polynomial  regression  equations.  The  univariate  polynomial 
regression  equations  described  the  functional  relationships  between  the  four  system 
timing  variables  for  each  of  the  22  separate  dependent  variables.  The  most  important 
variables  included  time  spent  looking  at  the  display,  time  spent  looking  at  the  keyboard 
while  typing,  typing  rate,  and  overall  operator  satisfaction.  The  multivariate  polynomial 
regression  analyses  provided  functional  relationships  in  terms  of  three  composite  mea¬ 
sures  representing  production,  waiting,  and  planning  activities  of  the  operator.  Although 
all  four  system  variables  were  significant  in  various  evaluations,  the  most  important 
system-timing  variables  across  all  analyses  were  the  system  response  times  and  keyboard 
echo  rates. 

Conclusions 


All  three  classes  of  metrics  (i.e.,  work  sampling*  embedded  performance  measures, 
and  operator  satisfaction  ratings)  are  needed  to  provide  a  complete  analysis  of  the  effects 
of  the  four  system  variables  on  operator  behavior.  By  using  these  three  classes  of 
measures  and  representing  the  functional  relationships  in  terms  of  response  surfaces,  the 
system  designer  can  easily  superimpose  the  various  surfaces  to  make  the  necessary 
human/computer  interface  design  tradeoffs.  Additionally,  a  more  general  interpretation 
of  the  human/computer  interface  can  be  made  by  using  multivariate  response  surfaces 
representing  operator  production,  waiting,  and  planning  activities. 
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INTRODUCTION 


Problem 


Navy  personnel  records  are  becoming  more  computerized,  both  to  increase  the 
productivity  of  personnelmen  who  enter  and  update  records  and  to  reduce  the  number  of 
data  entry  errors.  Two  current  Navy  systems,  the  Manpower,  Personnel,  and  Training 
Information  System  (MAPTIS)  and  the  Joint  Uniform  Military  Pay  System  (JUMPS),  are 
extremely  large-scale  information  management  systems  that  receive  widespread,  distrib¬ 
uted  entry  from  over  3000  field  offices.  Source  data  entry  to  each  system  is  extremely 
labor-intensive;  it  has  been  estimated  that  individuals  holding  personnelman  ratings  in 
Navy  personnel  offices  devote  about  25  percent  of  their  time  to  data  input  to  MAPTIS  and 
JUMPS  (Michna,  Laidlaw,  &  Oberm^yer,  1978). 

In  addition  to  the  investment  of  large  amounts  of  personnel  hours,  Obermayer  (1977) 
cites  two  other  critical  problem  areas:  significant  error  rates  (10-30%)  and  long  delays 
(70-90  days)  in  updating  personnel  information  entered  by  hand-typed  optical  character 
recognition  (OCR)  forms.  Significant  improvements  in  all  of  these  areas  are  feasible 
through  various  office  automation  procedures  involving  direct  human/computer  interface. 
However,  care  must  be  taken  to  consider  appropriate  human  engineering  design  principles 
to  optimize  the  human/computer  communication  interface.  The  magnitude  of  this  design 
problem  was  underscored  by  the  General  Accounting  Office,  in  a  1980  report  that 
evaluated  various  inefficiencies  in  the  Navy's  computerized  pay  system. 

Background 

Even  though  the  fundamental  concept  of  an  interactive  system  requires  a  continual 
interaction  between  the  human  and  the  computer,  few  data  exist  on  the  operation  of  the 
system  hardware  and  operator  behavior.  Although  each  system  is  somewhat  unique,  any 
on-line  interaction  with  time-sharing  systems  involves  several  factors.  Carbonell,  Elkind, 
and  Nickerson  (1968)  discussed  the  parameters  of  accessibility  and  response  time. 
Accessibility  is  the  ability  of  the  user  to  enter  the  time-sharing  system  and  is  a  function 
of  the  current  load.  Although  the  ideal  situation  would  be  a  time-sharing  system  that  is 
always  accessible  when  the  user  wants  it,  this  ideal  state  often  is  not  realized  and  no  data 
exist  on  the  effect  of  limited  accessibility  on  user  rates. 

Response  time,  on  the  other  hand,  is  the  amount  of  time  required  by  the  system  to 
respond  to  a  user  input  and  depends  on  a  variety  of  factors,  including  the  current  number 
of  users,  the  complexity  of  the  calculation  necessitated  by  the  user  input,  and  the 
system's  hardware  configuration.  If  the  response  time  of  an  interactive  system  is  not 
adequate,  the  human's  performance  may  deteriorate.  Obviously,  there  is  no  one  optimum 
response  time  that  pertains  to  all  time-sharing  situations.  In  fact,  Engel  and  Granda 
(1975)  present  guidelines  ranging  from  0.1  seconds  to  60  seconds  maximum  acceptable 
response  time,  depending  upon  the  system  recognized  activity  (e.g.»  key  response,  file 
update,  error  feedback)  and  user  activity  (e.g.,  system  activation,  loading,  and  restart). 
Generally,  the  recommended  guideline  for  system  acknowledgment  that  a  request  is  being 
processed  is  an  almost  instantaneous  response  time  (i.e.,  <  0.5  seconds).  Miller  (1968),  for 
example,  recommends  that  all  other  human/computer  interactions  should  have  less  than  a 
2-second  response  time  unless  the  operator  is  engaged  in  the  particular  terminal  operation 
only  infrequently. 

Actual  behavioral  data  of  the  effect  of  system  response  time  are  quite  limited. 
Morfieid,  Wiesen,  Grossberg,  and  Yntema  (1969)  studied  the  effect  of  response  times 
varying  from  1  to  100  seconds  on  user  problem-solving  performance.  The  average  time  to 
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completion  increased  as  expected.  However,  the  net  completion  time  also  increased, 
which  suggests  that  the  operator  was  becoming  distracted.  Additional  research  by 
Grossberg,  Wiesen,  and  Yntema  (1976)  introduced  unknown  variability  into  the  various 
response  times.  This  research  showed  that,  although  users  made  fewer  inquiries  of  the 
time-sharing  systems  with  longer  system  response  times,  system  delays  did  not  affect 
their  actual  time  to  solution. 

One  particularly  critical  issue  relating  to  the  effects  of  system  response  on  operator 
performance  is  that  much  of  the  previous  research  is  not  directed  toward  true  -ystem- 
related  variables  manipulated  within  realistic  operational  ranges.  The  current  data 
collection  effort  on  this  project  provided  some  meaningful  information  in  this  regard. 
Specifically,  variables  such  as  the  display  rate,  delays  in  displaying  echoing  of  keyboard 
inputs,  and  the  design  variables  vary  quite  markedly  in  existing  time-sharing  systems. 
Essentially  no  data  are  available  on  the  separate  and  combined  effects  of  these  variables 
on  operator  behavior.  System  and  display  design  decisions  are  constantly  being  made 
devoid  of  these  data  even  though  the  human  operator  is  the  ultimate  user  of  the 
interactive  system. 

A  preliminary  study  by  Beatty  and  Williges  (1979)  provided  the  background  data  for 
the  current  study.  Their  results  suggested  that  embedded  measures  of  the  operator's  data 
entry  performance  can  be  used  as  powerful  tools  in  measuring  the  human/computer 
interface.  In  this  regard,  both  user  ready  time  and  system  response  times  need  to  be 
evaluated  in  complicated  tasks  involving  personnel  transactions. 


A  more  comprehensive  approach  is  needed  where  a  variety  of  actual  system,  task, 
operator,  and  environment-independent  variables  are  manipulated  together  and  their 
functional  relationship  to  operator/analyst  performance  is  described.  With  the  inherent 
automatic  data  recording  capabilities  of  computer-based  systems,  this  approach  seems 
feasible.  Finkelman,  Wolfe,  and  Friend  (1977)  offer  polynomial  regression  as  a  reasonable 
method  to  define  such  functional  relationships  for  data  characterized  by  lower-order 
trends.  A  polynomial  expression  provides  a  convenient  approximation  to  a  variety  of 
mathematical  relationships,  thereby  making  it  a  powerful  tool  for  predicting  operator 
performance  while  still  using  a  standard  format.  The  general  form  of  such  a  second-order 
polynomial  model  would  be 
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where  human  behavior,  Y,  is  expressed  in  terms  of  an  intercept  value,  3q, 


(1) 

and  the 


weighted  linear  combinations  of  first-order  terms,  X.,  pure  quadratic  second-order  terms, 
X.2,  and  linear  interaction,  second-order  terms,  X.X^,  of  the  k  system  variables  stated  in 
terms  of  Xj's.  The  value  e  is  the  estimate  of  error  in  prediction.  Sample  estimates  of  the 


various  3  parameters  are  readily  obtained  through  standard  least-square  regression 

procedures. 


Recently,  Williges  (1977)  suggested  that  his  polynomial  regression  approach  would  be 
useful  in  developing  an  automated  assessment  scheme  of  personnel  performance  in 
computer-based  systems.  This  performance  scheme,  in  turn,  could  be  used  for  embedded 
performance  measurement,  evolutionary  system  operation,  performance  enhancement 
procedures,  and  the  development  of  realistic  data  bases  from  which  theoretical  extrapola¬ 
tions  can  be  made  to  the  design  of  future  human/computer  systems. 
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In  addition  to  specifying  the  system  parameters  (X.'s)  in  Equation  1,  one  must  also 

determine  the  appropriate  human  behavior  (Y).  The  embedded  performance  assessment 
discussed  by  Williges  (1977)  potentially  involves  a  variety  of  measures  dealing  with  time 
to  complete  a  task,  operator  waiting  times,  error  rates,  etc.,  that  can  be  automatically 
recorded  by  the  computer  system  while  the  operate-  is  using  the  interactive  terminal. 
However,  embedded  performance  measures  are  only  one  class  of  metrics  that  can  be  used 
to  evaluate  the  overall  human/computer  interface.  Other  classes  of  metrics  include  the 
human  operator's  subjective  ratings  of  satisfaction  with  the  system  configuration  and 
work  sampling  measures  estimating  the  proportion  of  time  spent  in  various  aspects  of  the 
interactive  human/computer  task.  Each  of  these  two  metrics  classes  have  been  used  only 
to  a  limited  extent  in  evaluating  operator  behavior  in  interactive  systems  (see,  for 
example,  Miller,  1977;  and  Hoecker  &  Pew,  1979). 


Objectives 


The  purpose  of  this  study  was  to  demonstrate  the  utility  of  incorporating  all  three 
classes  of  metrics  in  evaluating  human/ computer  interactions.  Each  class  of  measures 
was  used  separately  in  generating  functional  relationships  between  human  behavior  and 
four  systems  parameters.  The  resulting  functional  relationships  were  integrated  in  a 
multivariate  analysis  to  provide  an  overall  description  of  the  human/computer  interface. 


METHOD 


The  general  task  environment  was  structured  around  a  Navy  peronnel  records  task  in 
which  the  operator  was  required  to  use  an  interactive  computer  terminal  to  perform 
specified  transactions  on  simulated  personnel  records.  The  particular  transaction  used  in 
this  study  was  a  form-filling  task  analogous  to  a  pay  order  form  used  to  issue  a  temporary 
pay  change  for  a  given  individual.  Figure  1  depicts  the  display  layout  of  the  pay  order 
form  as  used  in  this  study.  Alphanumeric  information  was  entered  into  a  series  of  12 
fields  designated  on  the  display,  as  shown  in  Figure  1.  The  cursor  symbol  (>)  shown  at  the 
bottom  designated  a  working  area  of  the  display  used  for  query  language  commands. 
When  data  were  entered  in  any  field,  the  cursor  was  first  moved  to  that  field  to  activate 
the  area.  These  fields  included  information  items  such  as  date,  name,  social  security 
number,  duty  station,  amount  of  pay,  reason  for  change,  etc.  Specific  Navy  format  rules 
were  followed  for  entering  the  date,  name,  and  time  in  the  appropriate  fields  on  the 
interactive  terminal.  Even  though  all  records  used  in  this  experiment  were  simulated, 
they  did  represent  the  type  of  information  and  formatting  rules  used  in  actual  Navy 
personnel  records. 


Each  subject  was  required  to  perform  either  ADD  or  CHANGE  transactions  on  these 
records.  The  ADD  command  was  used  to  add  new  records  to  the  system,  whereas  the 
CHANGE  command  was  used  to  modify  existing  personnel  records.  All  information 
pertaining  to  the  revision  and  addition  to  records  was  presented  on  an  adjacent  plasma 
panel  via  a  PLATO  IV  terminal  (Bitzer  &  Johnson,  1971)  connected  to  the  University  of 
Illinois  PLATO  system.  The  presentation  of  these  ADD  and  CHANGE  requests  was  either 
structured  according  to  the  format  used  on  the  form-filling  interactive  display  or 
unstructured  in  a  free-flowing  text  format. 
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Figure  1.  Display  format  used  in  the  personnel  records  data  entry  task. 


The  arrangement  of  the  terminal  work  area  closely  followed  the  procedures  reported 
by  Beatty  and  Williges  (1979).  Figure  2  shows  this  arrangement,  which  consisted  of  two 
side-by-side  plasma  panels.  The  plasma  panel  on  the  right  was  the  PLATO  IV  terminal 
used  for  instructions  as  well  as  the  ADD  and  CHANGE  requests  during  data  entry  in  the 
actual  experimental  trials.  The  panel  on  the  left  was  a  special-purpose  terminal 
developed  by  Information  Technology  Limited  (ITL)  that  was  used  for  data  entry  in  the 
experiment.  This  display  projected  the  pay  order  form  shown  in  Figure  1  and  was  used 
interactively  by  the  subjects  in  the  form-filling  task.  A  one-way  communication  channel 
between  the  two  panels  called  the  next  data  entry  request  to  be  performed  at  the 
completion  of  the  preceding  request. 

Generic  Task  Simulation 

To  facilitate  the  experimental  evaluation  of  automated  performance  assessment  in  a 
personnel  records  task,  a  generic,  single-operator,  event-based  task  simulation  was 
developed.  The  hardware  for  this  system  is  a  512x512  parallel-plasma  panel  interfaced 
directly  to  a  laboratory  PDP  11/55  minicomputer.  The  parallel  display  panel  is  equipped 
with  both  a  32x32  touch  panel  entry  and  a  keyboard  input  capability  to  the  PDP  11/55 
computer.  The  computer  stores  the  simulated  personnel  records  for  the  performance 
assessment  task,  interprets  queries  made  by  the  subject  during  personnel  records 
transactions,  and  records  the  subject's  task  performance  in  terms  of  errors  and  response 
latencies.  These  performance  measures,  in  turn,  are  used  as  the  dependent  measures  in 
the  performance  assessment  profiles. 
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Figure  2.  Arrangement  of  the  interactive  computer  terminals  used 
in  the  data  entry  task. 


Two  general  software  routines  were  programmed  in  connection  with  the  generic  task 
simulation.  One  routine  allows  for  general-purpose  communication  between  the  PDP 
11/55  computer  and  the  parallel-plasma  panel.  This  set  of  assembly  language  routines 
enables  one  to  write  a  variety  of  alphanumeric  characters  on  the  panel  as  well  as  perform 
various  line  drawing  operations.  The  second  set  of  software  programs  was  developed  to 
generate  the  generic,  event-based  task.  These  programs  produce  a  table-driven  task 
simulation  that  allows  for  such  things  as  record  additions/deletions,  record  switching, 
page  switching,  field  switching,  updating,  and  a  primitive  command  language.  Details  on 
the  design  of  this  generic  task  simulation,  as  well  as  a  complete  source  list  of  the  various 
subroutines  are  provided  by  Mason,  Evans,  and  Beatty  (1979). 

Subjects 

Four  undergraduate  students  at  Virginia  Polytechnic  Institute  and  State  University 
(three  male  and  one  female)  were  used  as  subjects  in  this  experiment.  Subjects  had  no 
previous  experience  in  making  personnel  transactions  on  a  computer-based  system. 

Independent  Variables 

Four  parameters  relating  to  various  timing  parameters  of  the  computer  system  were 
manipulated:  system  delay  (SD),  display  rate  (DR),  echo  rate  (ER),  and  buffer  length  (BL). 
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Levels  of  each  of  these  independent  variables  were  set  according  to  results  of  pretest 
data,  effective  ranges  noted  in  the  scientific  literature,  and  realistic  ranges  encountered 
in  interactive  system  operation. 

The  SD  variable  controlled  the  delay  time  (in  seconds)  between  an  operator's  input 
command  (e.g.,  search  files,  next  field,  etc.)  and  the  computer  executions  of  that 
command  signified  by  returning  control  to  the  operator.  The  DR  variable  manipulated  the 
rate  (in  characters/second)  at  which  characters  were  displayed  on  the  screen  and  was 
somewhat  analogous  to  baud  rate  characteristics  of  standard  terminals.  The  last  two 
factors  were  both  related  to  keyboard  entry  timing.  ER  represented  the  delay  time  (in 
seconds)  between  a  keystroke  and  the  appearance  of  that  character  on  the  display  screen. 
BL  referred  to  the  number  of  characters  typed  on  the  keyboard  that  could  be  held  in  a 
buffer  memory  awaiting  display  on  the  interactive  plasma  panel. 

Experimental  Design 

To  provide  the  necessary  and  sufficient  data  to  solve  the  polynomial  expression 
stated  in  Equation  1  in  an  economical  fashion,  a  four-factor  central-composite  design  was 
used.  An  orthogonal  version  of  this  design  was  chosen  with  equal  replication  across  the 
entire  design  yielding  the  25  unique  treatment  combinations  of  five  levels  of  each  of  the 
four  independent  variables  shown  in  Table  1.  (See  Williges,  1980,  for  a  detailed 
description  of  the  development  and  use  of  central-composite  designs  in  behavioral 
research.)  The  linear  transformation  between  the  coded  values  of  the  central-composite 
design  and  the  real-world  values  of  the  four  systems  variables  are  summarized  in  Table  2. 

Each  subject  received  four  trials  on  each  of  the  resulting  25  treatment  combinations, 
thereby  yielding  a  within  subject-design.  The  four  trials  consisted  of  a  one-half  fractional 
replicate  of  the  combination  of  prompting  tone  (on  or  off),  a  trial  presentation  (structured 
or  unstructured  information),  and  a  task  type  (adding  a  record  or  changing  an  existing 
record).  The  third-order  interaction  was  used  as  the  defining  contrast  in  choosing  the 
one-half  replicate  such  that  two  subjects  received  one  of  the  resulting  replicates  and  the 
other  two,  the  other  replicate. 

Procedures 


Each  subject  received  a  computer-assisted  instruction  lesson  on  the  PLATO  terminal 
before  participating  in  the  experiment.  This  lesson  lasted  approximately  45  minutes  and 
provided  general  instruction  on  the  interactive  dispiay  used  in  the  experimental  sessions 
as  well  as  the  rules  for  listing  names,  dates,  and  times. 

Following  the  practice  session,  each  subject  participated  in  five  experimental 
sessions,  each  consisting  of  four  trials  on  five  treatment  combinations.  The  five 
treatment  combinations  were  chosen  randomly  for  each  subject.  Consequently,  each 
subject  was  required  to  complete  100  personnel  records  throughout  the  course  of  the 
experiment.  In  addition,  each  subject  received  four  practice  trials  in  the  first  experi¬ 
mental  session  to  become  familiar  with  the  experimental  protocol.  These  four  practice 
trials  included  the  +1.414  levels  of  all  factors  on  two  trials  and  the  -1.414  levels  of  all 
factors  on  the  other  two  trials,  thereby  showing  each  subject  the  possible  range  of 
treatment  conditions. 
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Table  1 


Coded  Values  of  Unique  Treatment  Combinations: 
A  Four-factor  Central-composite  Design 


Treatment 

Condition 

Independent  Variables 

System 

Delay 

Display 

Rate 

Echo 

Rate 

Buffer 

Length 

1 

+1 

+1 

+1 

+1 

2 

+1 

+1 

+  1 

-1 

3 

+  1 

+  1 

-1 

+1 

4 

+1 

+  1 

-1 

-1 

5 

+1 

-1 

+1 

+  1 

6 

+1 

-1 

+1 

-1 

7 

+1 

-I 

-1 

+1 

8 

+  1 

-1 

-1 

-1 

9 

-1 

+  1 

+  1 

+1 

-1 

+  1 

+  1 

-1 

11 

-1 

+  1 

-1 

+1 

12 

-1 

+1 

-1 

-1 

13 

-1 

-1 

+  1 

+1 

14 

-1 

-1 

+1 

-1 

15 

-1 

-1 

-1 

+  1 

16 

-1 

-1 

-1 

-1 

17 

+1.414 

0 

0 

0 

18 

-1.414 

0 

0 

0 

19 

0 

+1.414 

0 

0 

0 

-1.414 

0 

0 

21 

0 

0 

+1.414 

0 

22 

0 

0 

-1.414 

0 

23 

0 

0 

0 

+1.414 

24 

0 

0 

0 

-1.414 

25 

0 

0 

0 

0 

Table  2 

Linear  Transformations  Between  Coded  Values  Used 
in  the  Central-composite  Design  and  Real-world 
Levels  of  the  Four  System  Variables 


Levels  of  the  Four  Independent  Variables 


Variable 

'-T.TO 

— n - 

0 

+1 

+1.414 

System  Delay  (SD) 

0.10 

1.35 

5.05 

8.55 

10.00 

Display  Rate  (DR) 

240.00 

206.00 

125.00 

44.00 

10.00 

Echo  Rate  (ER) 

0.00 

0.22 

0.75 

1.28 

1.50 

Buffer  Length  (BL) 

1.00 

2.00 

4.00 

6.00 

7.00 

800S 


Dependent  Variables 

Three  general  classes  of  dependent  variables  were  measured  in  this  study:  work 
sampling,  embedded  performance  assessment,  and  operator  satisfaction  ratings.  As  shown 
in  Table  3,  several  specific  measures  were  collected  within  each  of  these  general  classes 
to  provide  a  total  of  22  dependent  variables. 


Table  3 

Classes  of  Dependent  Variables  Used  in  the 
Principal  Components  Analysis 


Class  Variables 


Work  Sampling  Looking  at  Information  (INF) 

Looking  at  Display  (DSP) 

Looking  at  Keyboard  (KBD) 
Information/Typing  (INF/TYP) 

Display /Typing  (DSP/TYP) 
Keyboard/Typing  (KBD/TYP) 

Embedded  Performance  Assessment  Typing  Rate  (TRATE) 

Field  Entry/User  Response  Time  (FE/URT) 
Next  Field/User  Response  Time  (NF/URT) 
Field  Entry/Ready  Time  (FE/RT) 

Next  Field/Ready  Time  (NF/RT) 

Ready  Responses  (RDRSP) 

Character  Erasures  (CHER) 

Checking  Time  (CKT) 

Satisfaction  Ratings  Tone  Rating  (TONR) 

System  Delay  Rating  (SDR) 

Display  Rate  Rating  (DSPR) 

Echo  Rate  Rating  (ERR) 

Buffer  Length  Rating  (BLR) 

Speed  Rating  (SPEED) 

Accuracy  Rating  (ACCUR) 

Overall  Rating  (OVER) 


Work  Sampling 

Throughout  the  entire  experimental  session,  a  closed-circuit  television  system  was 
used  to  monitor  the  time  spent  by  each  subject  on  various  aspects  of  the  personnel 
transcription  task.  The  overall  task  was  divided  into  six  mutually  exclusive  components. 
Three  of  these  components  dealt  with  viewing  information  on  either  the  PLATO  terminal 
(INF),  the  interactive  plasma  panel  display  used  in  the  data  entry  task  (DSP),  or  the  data 
entry  keyboard  (KBD).  The  other  three  components  were  concerned  with  typing  (data 
entries  while  viewing  either  the  input  information  (INF/TYP),  the  interactive  display 
(DSP/TYP),  or  the  keyboard  (KBD/TYP)).  Random  observations  were  made  throughout 
the  experimental  session  to  obtain  estimates  of  the  portion  of  task  time  devoted  to  each 


oi  these  six  categories.  The  mean  duration  between  samples  was  5  seconds,  and  the 
possible  durations  randomly  sampled  was  3,  4,  3, 6,  and  7  seconds  respectively. 

Embedded  Performance  Measires 

The  generic  task  simulation  allowed  for  on-line  data  collection  of  several  aspects  of 
operator  performance  while  using  the  interactive  terminal.  Specifically,  the  metering 
included  a  complete  transcription  of  keystroke  Inputs,  command  type,  and  a  variety  of 
performance  measures.  User  times  were  separated  into  response  time,  which  referred  to 
the  elapsed  time  from  a  computer  prompt  to  a  keystroke  input  and  ready  time,  which  is 
the  time  a  user  is  ready  to  make  an  input  but  the  computer  is  unable  to  respond.  The 
eight  embedded  performance  measures  used  in  this  study  included  the  operator's  typing 
rate  (TRATE),  the  user's  response  time  for  making  a  field  entry  (FE/URT),  the  user's 
response  time  for  selecting  the  next  field  (NF/URlT,  the  user's  ready  time  before  a  field 
entry  (FE/RT),  the  user's  ready  time  for  the  next  field  (NF/RT),  the  number  of  ready 
responses  ( RDRSP)  the  number  of  character  erasures  (CHER),  and  the  checking  time 
(CKT)  needed  to  ascertain  that  the  correct  record  was  chosed  from  the  data  base. 

Satisfaction  Ratings 

The  two  practice  trials  during  the  first  experimental  session  served  as  a  means  of 
anchoring  the  subject's  satisfaction  rating.  Following  each  set  of  four  trials  on  a 
particular  treatment  combination,  each  subject  was  required  to  complete  a  10-point, 
Likert-type  rating  scale  evaluating  the  prompting  tone  (TONR),  each  of  the  four 
independent  variables  (SDR,  DSPR,  ERR,  BLR),  and  operator  satisfaction  of  the  systems 
variables  on  spaed  (SPEED),  accuracy  (ACCUR),  and  overall  performance  (OVER).  The 
complete  list  of  questions  used  in  the  rating  scale  is  provided  In  Appendix  A. 


RESULTS  AND  DISCUSSION 

Both  univariate  and  multivariate  analyses  were  conducted  on  the  22  dependent 
variables  shown  in  Table  X  The  results  of  each  set  of  these  analyses  are  presented 
separately. 

Univariate  Analyses 

Before  evaluating  the  various  effects  of  the  system-timing  variables  manipulated  in 
this  study,  a  preliminary  analysis  was  conducted  on  the  fractional  replication  of  the  three 
control  variables  used  to  construct  the  data  entry  task  (i.e.»  the  alerting  tone,  the 
structuring  of  the  information,  and  the  entry  task  type).  Essentially,  there  were  no 
significant  differences  (g>  >  .03)  between  ADD  or  CHANGE  tasks  and  the  interactions  of 
these  control  variables  with  the  system-timing  variables.  Overall,  however,  the  presence 
of  the  tone  and  the  structuring  of  the  information  presented  to  the  subjects  had 
significant  effects  (£  <  .01)  on  the  percent  of  time  spent  viewing  the  display,  as  well  as 
user  response  times  during  data  entry.  Specifically,  the  alerting  tone  increased  the 
amount  of  time  spent  viewing  the  information  display  and  decreased  both  next  field  and 
field  entry  user  response  time.  Also,  as  expected,  the  unstructured  trials  caused  subjects 
to  spend  more  time  viewing  the  information  and  increased  the  field  entry  user  response 
times.  Since  the  control  variables  only  had  these  overall  effects,  the  trials  were 
combined  for  the  subsequent  univariate  and  multivariate  analyses. 

The  overall  analysis  pertains  to  the  three  metrics  of  satisfaction  ratings,  work 
sampling,  and  embedded  performance  measures.  In  each  dependent  variable  category, 
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second-order  polynomial  regression  equations  were  calculated  to  determine  the  functional 
relationship  between  a  specific  dependent  variable  and  the  four  system-timing  variables 
manipulated  in  this  experiment.  Standard  least-square  regression  procedures  were  used  to 
fit  these  polynomial  expressions.  Subsequently,  an  analysis  of  variance  (ANOVA)  was 
conducted  on  each  regression  analysis  to  isolate  the  statistically  significant  predictors. 
Comparisons  among  the  different  metrics,  therefore,  can  be  made  directly  in  terms  of  the 
differential  characteristics  of  the  various  polynomial  regression  equations.  In  addition,  a 
second-order,  orthogonal  design  was  used  so  that  the  partial  regression  weights  based  on 
coded  data  would  be  uncorrelated,  thereby  facilitating  the  interpretation  of  these  relative 
comparisons. 

Appendix  B  presents  a  complete  summary  of  each  of  the  separate  polynomial 
regression  equations,  as  well  as  the  subsequent  ANOVA.  A  listing  of  the  significant 
predictors  of  each  of  these  polynomial  regressions  is  presented  in  Table  4  for  easy 
reference.  The  interpretation  and  discussion  of  each  of  these  analyses  are  presented 
separately  by  class  of  metric. 


System-timing  variables  affect  the  amount  of  time  devoted  to  various  aspects  of  the 
task.  Overall,  Figure  3  shows  that  operators  spent  significantly  different  (g  <  .01) 
amounts  of  time  in  variqus  aspects  of  the  personnel  transaction  task.  The  two  largest 
proportions  of  time  were  spent  in  viewing  the  interactive  display  (DSP)  (26%)  and  viewing 
the  keyboard  while  entering  data  (KBD/TYP)  (28%). 

Subsequent  polynomial  repressions  of  the  time  spent  in  various  aspets  of  the  task  as  a 
function  of  system-timing  variables  showed  high  multipie  correlations  for  both  DSP  and 
KBD/TYP  (i.e.,  R2  =  .36  and  .34  respectively).  In  both  polynomial  regressions,  the  SD  and 
ER  variables  were  the  primary  predictors  of  work  sampling  time,  as  shown  in  Table  4. 
However,  the  effects  of  these  two  variables  were  quite  dissimiliar.  To  aid  in  interpreting 
these  differential  effects,  the  complete  second-order  polynomial  function  as  well  as  a 
transect  plot  of  DSP  and  KBD/TYP  performance  as  a  function  of  the  two  significant 
factors  SD  and  ER  are  shown  in  Figures  4  and  5  respectively.  The  other  two  system 
variables  are  held  constant  at  the  0  coding  level.  (Note  that  the  plots  represent  second- 
order  functions;  the  minor  perturbations  shown  in  the  figures  are  merely  artifacts  of  the 
particular  nearest  neighbor  algorithm  used  for  creating  the  plots  and  the  location  of 
predicted  data  points  across  the  surface.) 

Figure  4  shows  that  the  operator  spends  increasingly  more  time  viewing  the 
interactive  display  as  SD  and  ER  increase.  This  additional  viewing  time  is  necessary  both 
to  cross-check  echoing  of  a  typed  character  and  to  look  for  the  computer  prompt 
signifying  computer  availability  for  the  next  command  input.  Results  from  the  KBD/TYP 
prediction  e<pjation,  as  depicted  in  Figure  3,  show  opposite  effects;  namely,  the  operator 
spends  more  time  viewing  the  keyboard  and  typing  when  SD  and  ER  are  short,  thereby 
allowing  more  immediate  access  to  the  computer.  Additionally,  Figure  3  shows  that  the 
proportion  of  time  spent  on  KBD/TYP  decreases  rapidly  in  a  nonlinear  fashion  as  SD  and 
ER  increase. 

Embedded  Performance  Assessment 


The  summary  of  the  eight  measures  of  operator  performance  measures  provided  in 
Table  4  show  that  all  the  system-timing  variables  had  a  significant  (g  <  .01)  effect  on 
operator  performance  with  at  least  one  dependent  variable.  The  dependent  variable  with 
the  highest  multiple  correlation  in  the  regression  analysis  was  TRATE  (R*  » .61).  Two 
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Table  4 


Summary  of  Polynomial  Regression  ANOVAs 
for  Separate  Dependent  Variables 


Metric 

R2 

Significant  Predictors  (p<.01) 

Work  Sampling 

INF 

.044 

■— 

DSP 

.362 

ER,  SD,  DR2,  ERxSD 

KBD 

.076 

SD2 

INF/TYP 

.081 

SD 

DSP/TYP 

.091 

ER 

KBD/TYP 

.341 

ER,  SD,  ERxSD 

Embedded  Performance  Assessment 

TRATE 

.613 

BL,  ER,  ER2,  BLxER,  ERxDR 

FE/URT 

.233 

SD,  SD2 

NF/URT 

.082 

SD 

FE/RT 

.436 

SD,  SD2 

NF/RT 

.418 

SD,  SD2 

RDRSP 

.357 

SD 

CHER 

.063 

ER 

CKT 

.067 

DR 

Satisfaction  Ratings 

TONR 

.115 

ER 

SDR 

.530 

ER,  SD,  SD2,  ERxDR 

DSPR 

.141 

DR,  SD,  BL2,  ER2,  BLxDR 

ERR 

.510 

ER,  SD,  ER2,  SD2 

BLR 

.455 

ER,  SD,  BL2,  ER2,  SD2,  BLxER 

SPEED 

.586 

ER,  SD,  ER2,  SD2,  ERxSD 

ACCUR 

.464 

ER,  SD,  ER2,  SD2,  ERxSD 

OVER 

.519 

ER,  DR,  SD,  ER2,  SD2,  ERxDR,  ERxSD 

LOOKING  AT  DISPLAY  (DSP) 


DSP  =  0.2371  ♦  0.0017BL  ♦  0.0261ER  +  0.0028DR 

♦  0.0S31SD  -  0.0005BL*  ♦  0.0015ER*  +  0.0288DR1 

♦  0.0039SD*  ♦  0.0101  BL*ER  -  0.0135BL*DR 

-  0.0025BL*SD  +  0.0041ER*DR  -  0.0210ER*SD 

-  0.0054 DR*5D 


10.00 


Figure  4.  Response  surface  for  looking  at  display  as  affected  by 
echo  rate  and  system  delay. 


KEYBOARD/TYPING  (KBD/TYP) 


KBD/TYP  =  0.2751  +  0.0057BL  -  0.0558ER  +  0.001 9DR 

-  0.0768SD  -  0.0020BL2  +  0.01 14ER2  +  0.0123DR2 

-  0.0134SD2  -  0.0121BL*ER  -  0.0001BL*DR 

+  0.0153BL*SD  -  0.0069ER*DR  +  0.0233ER*SD 
+  0.0045DR*SD 


Figure  5.  Response  surface  for  keyboard/typing  as  affected  by  echo 
rate  and  system  delay. 


other  regression  analyses  dealing  with  operator  ready  times  (FE/RT  and  NF/RT)  also 
yielded  high  multiple  correlations  (R2  =  .46  and  .42,  respectively).  In  the  user  ready  time 
analyses,  the  SD  independent  variable  was  the  primary  predictor,  showing  that  operator 
ready  time  increased  as  system  delay  time  increased. 

The  typing  rate  analysis,  however,  resulted  in  no  significant  (g  <  .05)  effect  due  to 
SO.  Alternatively,  BL  rather  than  SD  combined  with  ER  as  the  primary  significant 
predictors  (g  <  .01).  The  resulting  perspective  response  surface  of  BL  and  ER  effects  on 
operator  typing  rate  is  shown  in  Figure  6.  Typing  rates  are  quite  low  when  only  one 
character  is  held  in  the  keyboard  buffer  and  echo  rates  of  typed  characters  are  delayed  by 
1.50  seconds.  In  this  situation,  the  operator  can  quickly  overtype  the  keyboard  buffer  due 
to  the  long  echo  delays  with  the  result  that  the  character  is  never  entered  into  the 


TYPING  RATE  (TRATE) 


TRATE  =  0.020  +  0.000BL  -  0.007ER  +  0.000DR 

-  0.000 SD  -  0.000BL2  +  0.001  ER2  -  0.000DR2 
+  0.000SD2  +  0.001  BL*ER  -  O.OOOBL*DR 

-  O.OOOBL#SD  -  0.001  ER*DR  -  O.OOOER*SD 

-  0.000 DR*  SD 


KEYBOARD  ECHO 
RATE  (ER) 


Figure  6.  Response  surface  for  typing  rate  as  affected  by  echo  rate 
and  system  delay. 


computer.  It  appears  that  this  particular  combination  of  system  variables  forces 
operators  to  slow  their  input  rates  to  match  the  slower  computer  system  characteristics. 
The  results  shown  in  Figure  6  also  suggest  that  the  short  buffer  length  can  easily  be 
compensated  for  by  short  echo  delays,  but  a  larger  buffer  length  does  not  compensate  for 
long  echo  delays  to  any  great  extent. 

Satisfaction  Ratings 

17)0  third  category  of  metrics  summarized  in  Table  4  deal  with  various  measures  of 
operator  satisfaction.  Ratings  of  satisfaction  with  the  separate  timing  variables  reflected 
significant  predictors  of  each  of  those  factors  in  the  polynomial  regression.  The  more 


important  rating  scales,  however,  dealt  with  SPEED,  ACCU,  and  OVER.  Surprisingly, 
these  three  ratings  were  highly  correlated  and  resulted  in  essentially  the  same  factional 
relationship  relating  the  systems  variables.  Namely,  the  significant  (j>  <  .01)  partial 
regression  weights  include  both  first-and  second-order  effects  of  system  delay  and 
keyboard  echo  rate  (i.e.,  SD,  ER,  SD2,  ER2,  and  SD*ER). 

To  summarize  this  effect,  the  polynomial  regression  and  transect  plot  of  the 
operator's  overall  rating  of  satisfaction  as  a  function  of  the  two  significant  factors  SD  and 
ER  are  shown  in  Figure  7.  The  other  two  system  variables  are  held  constant  at  the  0 
coding  level.  Clearly,  the  subjects  were  satisfied  (high  rating)  with  the  fastest  SD  and 
ER,  but  their  satisfaction  decreased  rapidly  as  timing  delays  were  introduced.  In  fact, 
Figure  7  shows  a  flat  plateau  of  almost  total  dissatisfaction  when  SD  was  greater  than  5 
seconds  and  ER  was  more  than  0.75  seconds  delayed. 


OVERALL  RATING  (OVER) 

OVER  =  2.390  +  0.190BL  -  1.774ER  -  0.305DR 

-  0.917SD  +  0.005BL2  +  1.068ER2  +  0.256DR2 
+  0.881SD2  +  0.109BL*ER  -  0.109BL*DR 

-  0.029BL*SD  +  0.328ER*DR  +  0.640ER*SD 
+  0.046DR*SD 


10.00 


Figure  7.  Response  surface  for  overall  rating  as  affected  by  echo 
rate  and  system  delay. 


Multivariate  Analyses 

Rather  than  consider  each  of  the  dependent  variables  separately,  one  can  consider 
combinations  of  these  measures,  which  define  necessary  and  sufficient  metric  classes 
needed  to  describe  human/computer  interactions.  For  example,  the  three  general  metrics 
used  in  the  univariate  analyses  are  quite  distinct  in  terms  of  behavioral  dimensions. 
Subsequent  multivariate  analyses  were  conducted  in  an  attempt  to  isolate  these  underly¬ 
ing  behavioral  dimensions.  First,  a  principal  component  analysis  was  conducted  to  cluster 
the  metrics.  The  resulting  dimension  score  from  the  principal  components  analysis  was 
then  used  as  the  dependent  variable  in  a  subsequent  polynomial  regression  analysis  to 
evaluate  the  functional  relationship  between  the  system-timing  variables  and  the 
behavioral  dimensions. 

Principal  Components  Analysis 

To  estimate  the  underlying  behavioral  dimensions,  a  principal  components  analysis 
was  conducted  on  21  of  the  dependent  variables  shown  in  Table  3.  To  avoid  the  problem 
of  oolinearity  in  the  work  sampling  data,  the  KBD  dependent  variable,  which  represented 
the  smallest  pecent  of  time,  was  eliminated.  The  dependent  variables  used  were  drawn 
from  the  major  independent  variables  in  each  of  the  three  metric  classes  of  work 
sampling,  embedded  performance  assessment,  and  satisfaction  ratings.  The  dependent 
measures  were  recorded  across  the  four  subjects  on  each  of  the  four  trials  of  the  resulting 
25  treatment  combinations  shown  in  Table  1,  thereby  resulting  in  a  21x400  matrix  for  the 
principal  components  analysis. 

The  results  of  the  principal  components  analysis  are  summarized  in  Table  5,  which 
shows  the  orthogonally-rotated  dimension  loadings  for  each  of  the  21  dependent  measures 
across  the  three  principal  components.  These  three  components  together  account  for  51.1 
percent  of  the  variance.  If  additional  dimensions  are  added,  the  percent  contribution 
drops  markedly.  Consequently,  the  three  dimensions  shown  in  Table  4  seem  to  describe 
the  clustering  most  parsimoniously.  These  three  clusters  seem  to  represent 
human/computer  interface  dimensions  of  operator  production,  waiting,  and  planning 
activities. 


By  using  the  orthogonally-rotated  weighting  matrix,  interpretation  of  the  principal 
components  analysis  is  facilitated.  As  shown  in  Table  5,  the  first  dimension,  which 
accounts  for  28.5  percent  of  the  variance,  is  most  heavily  weighted  on  typing  rate  and 
ratings  of  echo  rate,  buffer  length,  speed,  accuracy,  and  overall  satisfaction.  In  other 
words,  this  dimension  appears  to  be  related  to  production  activities  of  the  operator. 

The  second  dimension  accounts  for  13.3  percent  of  the  variance  and  appears  to  be 
representative  of  the  operator's  waiting  activities.  Metrics  such  as  time  spent  viewing 
the  display,  field  entry  and  next  field  ready  times,  operator  ready  responses,  and  ratings 
of  system  delay  weigh  most  heavily  on  the  operator's  waiting  dimension. 

Although  the  third  dimension  accounts  for  only  9.3  percent  of  the  variance,  it  does 
appear  to  represent  another  feature  of  the  human/computer  interface,  which  is  separate 
from  the  first  two.  Dependent  variables,  including  time  spent  viewing  the  display  while 
typing,  next  field  and  field  entry  user  response  times,  and  ratings  of  the  cueing  tone,  were 
the  primary  measures  clustered  on  this  dimension,  which  appears  to  be  related  to  planning 
activities.  Since  the  personnel  records  tasks  used  in  this  study  was  primarily  a 
transcription  task,  one  would  expect  planning  activities  to  account  for  only  a  small 
portion  of  the  operator's  performance.  In  other  human/computer  tasks,  this  activity  may 
become  much  more  important. 


Table  5 


Orthogonally-rotated  Factor  Pattern  of 
the  Principal  Components  Analysis 


Metric 

1 

(Production) 

Dimension 

2 

(Waiting) 

3 

(Planning) 

INF 

-0.028 

0.271 

0.371 

DSP 

-0.071 

-0.577 

-0.119 

INF/TYP 

0.098 

0.195 

0.092 

DSP/TYP 

-0.165 

-0.127 

-0.549 

KBD/TYP 

0.292 

0.399 

0.337 

TRATE 

0.744 

-0.143 

0.288 

FE/URT 

0.112 

0.273 

-0.599 

NF/URT 

0.087 

0.096 

-0.663 

FE/RT 

-0.117 

-0.870 

0.119 

NF/RT 

-0.069 

-0.878 

0.169 

RDRSP 

-0.033 

-0.751 

0.147 

CHER 

-0.261 

0.132 

0.071 

CKT 

0.038 

0.055 

-0.233 

SDR 

0.359 

0.738 

-0.092 

TONR 

0.195 

0.008 

0.759 

DSPR 

0.053 

0.311 

0.382 

ERR 

0.850 

0.151 

0.028 

BLR 

0.734 

0.194 

0.058 

SPEED 

0.863 

0.328 

-0.062 

ACCUR 

0.860 

0.352 

-0.010 

OVER 

0.876 

0.354 

-0.002 

Eigenvalues 

5.983 

2.798 

1.951 

Percent  of 

Total  Variance 

28.5% 

13.3% 

9.3% 

Multivariate  Response  Surfaces 

Each  of  the  three  composite  human/computer  interface  dimensions  (i.e.,  production, 
waiting,  and  planning)  were  used  separately  to  determine  the  functional  relationships 
among  the  system-timing  variables.  A  weighted  dimension  score  was  determined  for  each 
unrotated  dimension  and  was  used  as  the  dependent  variable  in  the  polynomial  regression 
analysis.  A  complete  second-order  polynomial  regression  was  calculated  to  predict 
production,  waiting,  and  planning  activities  as  a  function  of  the  four  system-timing 
variables.  The  results  of  those  analyses  are  summarized  in  Appendix  C. 

Even  though  each  of  the  three  regressions  had  significant  predictors,  the  prediction 
equations  for  both  production  and  waiting  activities  accounted  for  substantially  more 
variance  (R2  =  .623  and  .517  respectively)  than  the  prediction  of  planning  activities 
(R2  =  .162).  Although  comparisons  are  presented  among  all  three  dimensions  for 
completeness,  the  low  multiple  correlation  coefficient  for  planning  makes  interpretation 
of  this  dimension  somewhat  suspect. 
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Linear  and  quadratic  effects  of  system  response  time  and  keyboard  echo  rates  were 
the  primary  predictors  of  production  and  planning  activities  ([>  <  .001),  whereas  the  linear 
effects  of  all  four  timing  variables  and  the  quadratic  effect  of  keyboard  echo  rates  were 
the  main  significant  predictors  of  waiting  activities  (|>  <  .05).  To  illustrate  the  dif¬ 
ferential  (Effects  of  the  system-timing  variables,  perspective  response  surfaces  of 
operator  production,  waiting,  and  planning  activities  are  shown  in  Figures  8,  9,  and  10 
respectively,  with  buffer  length  and  display  rate  held  constant  at  the  mean  levels.  By 
comparing  these  figures,  one  can  see  that  system  delay  and  keyboard  echo  rate  were 
important  predictors  of  operator  activities,  but  these  variables  affected  operator  behavior 
differentiaily.  Figure  8  shows  production  activity  to  be  highest  at  the  shortest  system 
delay  and  keyboard  echo  rate.  As  delays  in  either  of  these  two  system  timing  variables 
increase,  production  activity  decreases  markedly.  On  the  other  hand,  waiting  activities, 
as  shown  in  Figure  9,  are  lowest  when  the  system  delay  is  shortest  and  keyboard  echo  rate 
is  the  longest.  In  this  case,  the  long  keyboard  echoing  rates  mask  the  system  response 
time  effects  because  the  operators  cannot  make  ready  responses  quickly.  Other 
differences  are  due  to  the  significant  buffer  length  and  display  rate  effects,  which 
increase  waiting  activities.  Finally,  Figure  10  shows  a  marked  curvilinear  effect  of 
system  delay  such  that  planning  activities  are  reduced  at  an  intermediate  system  delay 
and  increase  at  extremely  slow  and  fast  system  delays. 

PRODUCTION  (P) 

P  =  -  0.43  +  0.07BL  -  0.57ER  -  0.05DR  -  0.62SD  -  0.02BL2 

+  0.30ER2  +  0.07DR2  +  0.19SD2  +  0.06BL*ER  -  0.06BL*DR 
-  0.07BL*SD  +  0.05ER*DR  +  0.13ER*SD  -  0.02DR*SD 


Figure  8.  Response  surface  for  production  as  affected  by  echo  rate 
and  system  delay. 


WAITING  (W) 

W  =  -  0.15  +  0.13BL  -  0.54ER  +  0.08DR  +  0.55SD  -  0.08BL2 

+  0.13ER2  +  0.06DR2  +  0.07SD2  +  0.04BL*ER  -  0.09BL*DR 
-  0.06BL*SD  -  0.07ER*DR  +  0.02ER*SD  -  0.04DR*SD 


R2  =  .517 


Figure  9.  Response  surface  for  waiting  as  affected  by  echo  rate  and 
system  delay. 


PLANNING  (PL) 


PL  =  -  0.43  +  0.03BL  -  0.23ER  -  0.07DR  +  0.26SD  +  0.04BL2 

+  0.10ER2  +  0.03DR2  +  0.36SD2  +  0.01BL*ER  +  0.01BL*DR 
+  0.05BL*SD  +  0.05ER*DR  -  0.03ER*SD  +  O.IIDR*SD 


Figure  10.  Response  surface  for  planning  as  affected  by  echo  rate 
and  system  delay. 
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Composite  Multivariate  Surface 


Tradeoffs  among  the  three  multivariate  surfaces  can  be  made  by  superimposing  the 
surfaces  to  form  a  composite  multivariate  surface.  These  composite  surfaces  can  be 
constructed  in  a  variety  of  ways,  depending  upon  the  weightings  chosen  for  the  separate 
surfaces.  Two  of  these  alternatives  are  shown  in  Figure  11,  which  depicts  the  composite 
of  production  activities  added  to  the  inverse  of  waiting  and  planning  activities.  These 
composite  surfaces  then  represent  combined  throughput  where  the  high  score  equals  high 
production  and  low  waiting  and  planning  activities. 

Figure  11a  depicts  the  composite  surface  based  on  equal  and  additive  contributions  of 
the  three  separate  activities;  and  Figure  lib,  a  composite  surface  based  on  differential 
contributions  of  the  three  separate  activities.  Specifically,  the  differential  contributions 
in  Figure  lib  are  determined  by  the  percent  of  variance  accounted  for  by  the  production, 
waiting,  and  planning  dimensions  (i.e.,  28.5,  13.3,  &  9.3  respectively).  By  comparing 
Figures  11a  and  b,  one  can  see  that  these  two  strategies  result  in  slightly  different 
composite  surfaces.  When  the  three  activities  are  combined  in  an  additive  manner  (Figure 
1  la),  the  composite  surface  is  almost  a  rising  plain  that  is  dominated  by  SD.  On  the  other 
hand,  when  the  composite  surface  is  based  on  percent  of  variance  (Figure  lib),  it  appears 
more  characteristic  of  the  production  activity  surface,  which  is  weighted  most  heavily  in 
the  composite.  Clearly,  one  must  carefully  consider  the  weighting  alternatives  to 
generate  the  composite  surface  most  appropriate  for  a  particular  system  applications. 


CONCLUSIONS 

Clearly,  all  three  classes  of  metrics  are  needed  to  provide  a  complete  analysis  of  the 
effects  of  the  four  systems  variables  on  operator  behavior.  By  choosing  any  of  these 
metric  classes,  only  part  of  the  description  of  operator  behavior  is  available.  The 
different  metric  classes  not  only  show  different  functional  relationships  for  the  same 
system  variables,  but  also  that  different  system  variables  are  primary  determinants  of 
operator  behavior  in  different  metric  classes.  By  using  these  three  classes  of  measures 
and  representing  the  functional  relationships  in  terms  of  response  surfaces,  the  system 
designer  can  easily  superimpose  the  various  surfaces  to  make  the  necessary 
human/ computer  interface  design  tradeoffs. 

Besides  using  the  separate  dependent  measures  to  determine  specific  system  design 
considerations,  the  multivariate  response  surfaces  allow  for  a  more  general  interpretation 
of  the  human/computer  interface.  These  multivariate  analyses  represent  operator 
behavior  at  the  human/computer  interface  in  terms  of  three  major  activities-- production, 
waiting,  and  planning.  In  the  personnel  records  task  used  in  this  study,  the  planning  aspect 
of  operator  activities  was  not  central  and  accounted  for  only  a  small  percent  of  variance. 
Additional  research  is  needed  to  determine  if  these  same  three  activities  characterize 
human  performance  in  a  variety  of  computer  tasks  with  differential  weightings  of  these 
dimensions  across  tasks. 
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OPERATOR  SATISFACTION  RATING  SCALE 
DIRECTIONS:  Circle  the  appropriate  response  for  the  "YES”  and  "NO"  questions. 

1.  TONE 

a.  Did  the  tone  affect  your  performance? 

Yes  No 

b.  Use  a  slash  to  indicate  whether  the  Tone  was  OK,  interfering,  or  helpful. 


INTERFERING  OK  HELPFUL 

2.  SYSTEM  RESPONSE  TIME 


System  respone  time  is  the  time  it  takes  the  computer  to  respond  to  your 
commands.  It  is  measured  from  the  time  you  hit  the  NEXT  key  until  the  time 
you  see  the  "command  arrow." 

a.  Did  the  system  response  time  affect  your  performance? 

Yes  No 

b.  Use  a  slash  to  indicate  whether  the  system  response  time  was  OK,  too  slow, 
or  too  fast. 


TOO  SLOW  OK  TOO  FAST 


3.  DYNAMIC  DISPLAY  RATE 


Dynamic  display  rate  is  the  speed  with  which  the  computer  writes  the  DATA  in  a 
field  (such  as  the  Name  Field)  when  you  retrieve  a  record  from  the  file  (for 
example,  on  a  change). 

a.  Did  the  dynamic  display  rate  affect  your  performance? 

Yes  No 

b.  Use  a  slash  to  indicate  whether  the  Dynamic  Display  Rate  was  OK,'  too  slow, 
or  too  fast. 


+ - + - + - + - + - + - + - + - + - + - + - +- — + — -+ - + - + 


TOO  SLOW 


OK 


TOO  FAST 


KEVSTROKE  ECHO  DELAY 


Keystroke  echo  delay  is  the  time  it  takes  the  computer  to  write  a  character  on 
the  display  after  you  have  made  a  keystroke. 

a.  Did  the  keystroke  echo  delay  affect  your  performance? 

Yes  No 

b.  Use  a  slash  to  indicate  whether  the  keystroke  echo  delay  was  OK,  too  long, 
or  too  short. 


TOO  LONG  OK  TOO  SHORT 

TYPE  AHEAD  BUFFER  LENGTH 


Type  ahead  buffer  length  is  the  number  of  characters  you  can  type  ahead  of  what 
you  can  see  on  the  display. 

a.  Did  the  length  of  the  type  ahead  buffer  affect  your  performance? 

Yes  No 

b.  Use  a  slash  to  indicate  whether  the  type  ahead  buffer  was  OK,  too  short,  or 
too  long. 


+ + + + — -+ + + — — + + + — — + — — + 

TOO  LONG  OK  TOO  SHORT 


OPERATOR  SATISFACTION:  SPEED 

Are  you  satisfied  that  the  characteristics  of  this  system  did  not  slow  down  your 
completion  of  the  task? 


+ +- +- + + - + — — +— — 4— + + +  — — + -+• — — + — — + 

TOTALLY  UNSATISFIED  TOTALLY  SATISFIED 


THE  SYSTEM  ALWAYS  SLOWED 
DOWN  MY  PERFORMANCE. 


THE  SYSTEM  NEVER  SLOWED 
DOWN  MY  PERFORMANCE. 


OPERATOR  SATISFACTION:  ACCURACY 

Are  you  satisfied  that  the  characteristics  of  this  system  did  not  decrease  your 
accruacy  in  completing  the  task? 

TOTALLY  UNSATISFIED  TOTALLY  SATISFIED 

THE  SYSTEM  ALWAYS  THE  SYSTEM  NEVER 

DECREASED  MY  ACCURACY.  DECREASED  MY  ACCURACY. 

OPERATOR  SATISFACTION:  OVERALL 

Are  you  satisfied  that  the  characteristics  of  this  system  did  not  interfere  with 
your  overall  performance? 

TOTALLY  UNSATISFIED  TOTALLY  SATISFIED 


THE  SYSTEM  ALWAYS 
DECREASED  MY  PERFORMANCE 


THE  SYSTEM  NEVER 
DECREASED  MY  PERFORMANCE 


APPENDIX  B 

ANALYSIS  OF  VARIANCE  SUMMARY  TABLES 
UNIVARIATE  ANALYSES 


B-0 


Table  B-l 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  INF 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

B 

Regression 

(14) 

0.15550 

1.27 

0.2253 

Buffer  Length  (BL) 

1 

0.00471 

0.54 

0.4639 

Echo  Rate  (ER) 

1 

0.00277 

0.32 

0.5741 

Display  Rate  (DR) 

1 

0.00010 

0.01 

0.9132 

System  Delay  (SD) 

1 

0.01843 

2.10 

0.1479 

BL2 

1 

0.03395 

3.87 

0.0498 

ER2 

1 

0.00004 

0.01 

0.9406 

DR2 

1 

0.00005 

0.01 

0.9390 

SD2 

1 

0.02183 

2.49 

0.1154 

BL*ER 

1 

0.00000 

0.00 

0.9877 

BL*DR 

1 

0.01476 

1.68 

0.1952 

BL*SD 

1 

0.04255 

4.85 

0.0282 

ER*DR 

1 

0.00075 

0.09 

0.7699 

ER*SD 

1 

0.01528 

1.74 

0.1875 

DR*SD 

1 

0.00023 

0.03 

0.8710 

Residual 

385 

3.37565 

Total 

399 

3.53115 

Second-order  Polynomial  Regression 


INF  =  0.1673  +  0.0038BL  -  0.0029ER  -  0.0005DR 

-  0.0075SD  -  0.01 62BL2  -  0.0006ER2  -  0.0006DR2 
+  0.0130SD2  -  0.000 1BL*ER  -  0.0075BL*DR 

-  0.0128BL*SD  +  0.0017ER*DR  -  0.0077ER*SD 

-  0.0009DR*SD 


R2  =  .044 


Table  B-2 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  DSP 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

£ 

Regression 

(14) 

2.74673 

15.58 

0.0001 

Buffer  Length  (BL) 

1 

0.00096 

0.08 

0.7817 

Echo  Rate  (ER) 

1 

0.21954 

17.44 

0.0001 

Display  Rate  (DR) 

1 

0.00256 

0.20 

0.6523 

System  Delay  (SD) 

1 

2.21472 

175.91 

0.0001 

BL2 

1 

0.00003 

0.00 

0.9552 

ER2 

1 

0.00031 

0.03 

0.8742 

DR2 

1 

0.10655 

8.46 

0.0038 

SD2 

1 

0.00203 

0.16 

0.6876 

BL*ER 

1 

0.02625 

2.08 

0.1496 

BL*DR 

1 

0.04696 

3.73 

0.0542 

BL*SD 

1 

0.00168 

0.13 

0.7146 

ER*DR 

1 

0.00446 

0.36 

0.5516 

ER*SD 

1 

0.11290 

8.97 

0.0029 

DR*SD 

1 

0.00718 

0.61 

0.4341 

Residual 

385 

4.84727 

Total 

399 

7.59400 

Second-order  Polynomial  Regression 


DSP  =  0.2371  +  0.0017BL  +  0.0261ER  +  0.0028DR 

+  0.0831  SD  -  0.0005BL2  +  0.001 5ER2  +  0.0288DR2 
+  0.0039SD2  +  0.0101BL*ER  -  0.0135BL*DR 

-  0.0025BL*SD  +  0.0041ER*DR  -  0.0210ER*SD 

-  0.0054 DR* SD 


Table  B-3 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  KBD 


ANOVA  Summary  Table 

Source 

df 

SS 

F 

P 

Regression 

(14) 

0.03078 

2.26 

0.0057 

Buffer  Length  (BL) 

1 

0.00000 

0.00 

0.9736 

Echo  Rate 

(ER) 

1 

0.00073 

0.76 

0.3847 

Display  Rate  (DR) 

1 

0.00377 

3.89 

0.0494 

System  Delay  (SD) 

1 

0.00512 

5.27 

0.0223 

BL2 

1 

0.00063 

0.66 

0.4191 

ER2 

1 

0.00045 

0.47 

0.4955 

DR2 

1 

0.00384 

3.95 

0.0475 

SD2 

1 

0.00874 

8.99 

0.0029 

BL*ER 

1 

0.00272 

2.81 

0.0948 

BL*DR 

1 

0.00442 

4.55 

0.0336 

BL*SD 

1 

0.00023 

0.24 

0.6216 

ER*DR 

1 

0.00003 

0.03 

0.8537 

ER*SD 

1 

0.00002 

0.03 

0.8659 

DR*SD 

1 

0.00001 

0.02 

0.8950 

Residual 

385 

0.37440 

Total 

399 

0.40518 

Second-order  Polynomial  Regression 

KBD 

0.0281  +  0.0017BL  -  0.0015ER  - 

0.0034  DR 

+  0.0040SD 

f  0.0022BL2  -  0.0018ER2  -  0.0054DR2 

-  0.0082SD1 

-  0.0032BL*ER  +  0.0041BL*DR 

-  0.0009BL* 

SD  -  0.0003ER*DR 

-  0.0003ER*SD 

-  0.0002DR*SD 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

R 

Regression 

(14) 

0.10757 

2.42 

0.0029 

Buffer  Length  (BL) 

1 

0.00472 

1.49 

0.2236 

Echo  Rate(ER) 

1 

0.01213 

3.82 

0.0514 

Display  Rate  (DR) 

1 

0.00411 

1.30 

0.2556 

System  Delay  (5D) 

1 

0.03059 

9.63 

0.0021 

BL2 

1 

0.00787 

2.48 

0.1160 

ER2 

1 

0.00325 

1.02 

0.3126 

DR2 

1 

0.01987 

6.26 

0.0128 

SD2 

1 

0.00176 

0.55 

0.4570 

BL*ER 

1 

0.00228 

0.72 

0.3969 

BL*DR 

1 

0.00151 

0.48 

0.4908 

BL*SD 

1 

0.00013 

0.04 

0.8371 

ER*DR 

1 

0.00040 

0.13 

0.7207 

ER*SD 

1 

0.01422 

4.48 

0.0350 

DR*SD 

1 

0.00468 

1.47 

0.2254 

Residual 

385 

1.22301 

Total 

399 

1.33059 

Second-order  Polynomial  Regression 

INF/TYP  =  0.0498  +  0.0038BL  -  0.0061  ER  +  0.0035DR 

-  0.0097SD  -  0.0078BL2  -  0.0050ER2  -  0.0124DR2 

-  0.0037SD2  +  0.00293L*ER  +  0.0024BL*DR 

-  0.0007BL*SD  +  0.0012ER*DR  +  0.0074ER*SD 

-  0.0042DR*SD 


R2  =  .080 


Table  B-5 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  DSP/TYP 


ANOVA  Summary  Table 


Source 

Regression 
Buffer  Length  (BL) 
Echo  Rate  (ER) 
Display  Rate  (DR) 
System  Delay  (SD) 
BL2 
ER2 
DR2 
SD2 
BL*ER 
BL*DR 
BL*SD 
ER*DR 
ER*SD 
DR*SD 

Residual 

Total 


df 

SS 

(14) 

0.77597 

1 

0.07464 

1 

0.51951 

1 

0.00601 

1 

0.01594 

1 

0.00984 

1 

0.00385 

1 

0.06548 

1 

0.00894 

1 

0.00442 

1 

0.05505 

1 

0.00083 

1 

0.00000 

1 

0.00082 

1 

0.01056 

385 

7.74747 

399 

8.52345 

F 

E 

2.75 

0.0007 

3.71 

0.0548 

25.82 

0.0001 

0.30 

0.5848 

0.79 

0.3739 

0.49 

0.4844 

0.19 

0.6621 

3.25 

0.0720 

0.44 

0.5053 

0.22 

0.6395 

2.74 

0.0989 

0.04 

0.8384 

0.00 

0.9846 

0.04 

0.8393 

0.52 

0.4692 

Second-order  Polynomial  Regression 


DSP/TYP  =  0.2424  -  0.01 52BL  +  0.0402ER  -  0.0043DR 

+  0.0070SD  +  0.0087BL2  -  0.0054ER2  -  0.0226DR2 
+  0.0083SD2  -  0.004 1BL*ER  +  0.0146BL»DR 
+  0.0018BL*SD  +  0.0001ER*DR  -  0.0017ER*SD 
+  0.0064DR*SD 


R2  =  .091 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

R 

Regression 

(14) 

3.21812 

14.25 

0.0001 

Buffer  Length  (BL) 

1 

0.0107 

0.67 

0.4150 

Echo  Rate  (ER) 

1 

0.99864 

61.90 

0.0001 

Display  Rate  (DR) 

1 

0.00118 

0.07 

0.7865 

System  Delay  (SD) 

1 

1.89195 

117.28 

0.0001 

BL2 

1 

0.00052 

0.03 

0.8570 

ER2 

1 

0.01680 

1.04 

0.3081 

DR2 

1 

0.01951 

1.21 

>.2719 

SD2 

1 

0.02314 

1.43 

0.2317 

BL*ER 

1 

0.03762 

2.33 

0.1275 

BL*DR 

1 

0.00000 

0.00 

0.9881 

BL*SD 

1 

0.06015 

3.73 

0.0542 

ER*DR 

1 

0.01243 

0.77 

0.3806 

ER*SD 

1 

0. 14006 

8.68 

0.0034 

DR*SD 

1 

0.00531 

0.33 

0.5662 

Residual 

385 

6.21102 

Total 

399 

9.42914 

Second-order  Polynomial  Regression 

KBD/TYP  =  0.2751 

+  0.0057BL  - 

0.0558ER  +  0.001 9DR 

-  0.0768SD  -  0.0020BL2  +  0.01 14ER2  +  0.0123DR* 

-  0.0134SD2  -  0.0121BL*ER  -  0.0001BL*DR 

+  0.0153BL*SD  -  0.0069ER*DR  +  0.0233ER*SD 
+  0.0045DR*5D 


R2  =  .341 


Table  B-7 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  TRATE 


ANOVA  Summary  Table 


Source 

Regression 
Buffer  Length  (BL) 
Echo  Rate  (ER) 
Display  Rate  (DR) 
System  Delay  (SD) 
BL2 
ER2 
DR2 
SD2 
BL*ER 
BL*DR 
BL*SD 
ER*DR 
ER*SD 
DR*SD 

Residual 

Total 


df 

SS 

(14) 

0.01841 

1 

0.00030 

1 

0.01603 

1 

0.00014 

1 

0.00000 

1 

0.00012 

1 

0.00041 

1 

0.00000 

1 

0.00001 

1 

0.00049 

1 

0.00013 

1 

0.00019 

1 

0.00041 

1 

0.00000 

1 

0.00012 

385 

0.01161 

399 

0.03002 

F 

R 

43.60 

0.0001 

10.06 

0.0016 

531.64 

0.0001 

4.69 

0.0309 

0.01 

0.9240 

4.14 

0.0426 

13.90 

0.0002 

0.00 

0.9543 

0.34 

0.5579 

16.54 

0.0001 

4.50 

0.0345 

6.45 

0.0115 

13.70 

0.0002 

0.33 

0.5666 

4.17 

0.0419 

Second-order  Polynomial  Regression 


TRATE  =  0.020  +  0.000BL  -  0.007ER  +  O.OOODR 

-  0.0000SD  -  0.000BL2  +  0.001  ER2  -  0.0000DR2 
+  0.000SD2  +  0.001  BL*ER  -  O.OOOBL*DR 

-  0.000BL*SD  -  0.001ER*DR  -  0.000ER*SD 

-  0.000DR*SD 


R2  =  .613 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

e 

Regression 

(14) 

305970.28 

9.32 

0.0001 

Buffer  Length  (BL) 

1 

145.03 

0.06 

0.8037 

Echo  Rate  (ER) 

1 

552.28 

0.24 

0.6278 

Display  Rate  (DR) 

1 

6907.27 

2.95 

0.0869 

System  Delay  (SD) 

1 

228921.60 

97.60 

0.0001 

BL2 

1 

37.09 

0.02 

0.9013 

ER2 

1 

3969.50 

1.69 

0.1941 

DR2 

1 

5626.22 

2.40 

0.1222 

SD2 

1 

40835.15 

17.41 

BL*ER 

1 

6045.06 

2.58 

BL*DR 

1 

1914.06 

0.82 

0.3669 

BL*SD 

1 

1207.56 

0.51 

0.4735 

ER*DR 

1 

2013.76 

0.86 

ER*SD 

1 

968.76 

0.41 

0.5208 

DR*SD 

1 

6826.89 

2.91 

0.0888 

Residual 

385 

902983.09 

Total 

399 

1208953.37 

Second-order  Polynomial  Regression 


FE/URT  =  44.924  +  0.673BL  -  1.31 3ER  -  4.646DR 

-  26.747SD  +  0.531  BL2  +  5.563ER2  +  6.626DR2 
+  17.864SD2  -  4.859BL*ER  -  2.734BL*DR 
+  2.171BL*SD  +  2.804ER*DR  -  1.945ER*SD 
+  5.164DR*SD 


R2  =  .253 


Table  B-9 

Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  NF/URT 


ANOVA  Summary  Table 


NF/URT  =  51.508  -  2.104BL  +  2.955ER  -  2.388DR 

-  7.652SD  -  2.310BL2  +  2.456ER2  -  3.436DR2 
+  6.942SD2  +  0.187BL*ER  -  0.648BL*DR 
+  0.945BL*SD  +  2.671  ER*  DR  -  0.859ER*SD 
+  2.757DR*SD 


R2  =  .081 


Table  B-10 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  FE/RT 


Source 

Regression 

Buffer  Length  (BL) 

Echo  Rate  (ER) 

Display  Rate  (DR) 

System  Delay  (SD) 

BL2 

ER2 

DR2 

SD2 

BL*ER 

BL*DR 

BL*SD 

ER*DR 

ER*SD 

DR*SD 

Residual 

Total 


ANOVA  Summary  Table 


df 

SS 

(14) 

4247357.34 

1 

45327.41 

1 

5749.53 

1 

62538.16 

1 

3860652.69 

1 

1440.58 

1 

14990.31 

1 

7424.57 

1 

108013.11 

1 

27163.16 

1 

71.19 

1 

46359.47 

1 

201.28 

1 

1985.81 

1 

65440.03 

385 

5069023.36 

399 

9316380.71 

F 

£ 

23.04 

0.0001 

3.44 

0.0643 

0.44 

0.5091 

4.75 

0.0299 

293.22 

0.0001 

0.11 

0.7405 

1.14 

0.2862 

0.56 

0.4536 

8.20 

0.0044 

2.06 

0.1517 

0.01 

0.9414 

3.52 

0.0613 

0.02 

0.9017 

0.15 

0.6980 

4.97 

0.0264 

Second-order  Polynomial  Regression 


FE/RT  =  88.889  +  11.901BL  -  4.238ER  +  13.980DR 

+  109.842SD  -  3.361  BL2  -  10.832ER2  +  7.610DR2 
+  29.054 SD2  -  10.300BL*ER  -  0.527BL*DR 
+  13.457BL*SD  +  0.886ER*DR  -  2.785ER*SD 
+  15.988DR*SD 


R2  =  .455 


I 


B-10 


Table  B-ll 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  NF/RT 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

£ 

Regression 

(14) 

3316465.67 

19.78 

0.0001 

Buffer  Length  (BL) 

1 

23215.30 

1.94 

0.1646 

Echo  Rate  (ER) 

1 

22553.15 

1.88 

0.1707 

Display  Rate  (DR) 

1 

12233.77 

1.02 

0.3127 

System  Delay  (SD) 

1 

3024378.77 

252.59 

0.0001 

BL2 

1 

162.70 

0.01 

0.9079 

ER2 

1 

655.53 

0.06 

0.8145 

DR2 

1 

47.07 

0.00 

0.9494 

SD2 

1 

148639.03 

12.41 

0.0005 

BL*ER 

1 

36409.41 

3.04 

0.0820 

BL*DR 

1 

190.78 

0.02 

0.8996 

BL*SD 

1 

19757.81 

1.65 

0.1997 

ER*DR 

1 

1093.12 

0.09 

0.7627 

ER*SD 

1 

12953.28 

1.08 

0.2989 

DR*SD 

1 

14175.67 

1.18 

0.2772 

Residual 

385 

4609797.72 

Total 

399 

7926263.39 

Second-order  Polynomial  Regression 

NF/RT  =  65.470  +  8.517BL  -  8.395ER  +  6.183DR 

+  97.220SD  +  1.120BL2  -  2.271ER*  -  0.614DR2 
+  34.083SD2  -  1 1.925BL*ER  +  0.863BL*DR 
+  8.785BL*SD  -  2.066ER*DR  -  7.113ER*SD 
+  7.441  DR*SD 


R2  =  .418 


Table  B-12 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  RDRSP 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

R 

Regression 

(14) 

10810.53 

15.27 

0.0001 

Buffer  Length  (BL) 

l 

51.89 

1.03 

0.3117 

Echo  Rate  (ER) 

1 

8.88 

0.18 

0.6754 

Display  Rate  (DR) 

1 

6.10 

0.12 

0.7284 

System  Delay  (SD) 

1 

10075.54 

199.26 

0.0001 

BL2 

1 

168.90 

3.34 

0.0685 

ER2 

1 

114.38 

2.26 

0.1334 

DR2 

1 

92.77 

1.84 

0.1763 

SD2 

l 

6.58 

0.13 

0.7184 

BL*ER 

1 

59.09 

1.17 

0.2803 

BL*DR 

1 

29.56 

0.58 

0.4449 

BL*SD 

1 

75.47 

1.49 

0.2226 

ER*DR 

1 

103.78 

2.05 

0.1528 

ER*SD 

1 

17.53 

0.35 

0.5563 

DR*SD 

1 

0.00 

0.00 

0.9930 

Residual 

385 

19467.21 

Total 

399 

30277.75 

Second-order  Polynomial  Regression 

RDRSP  =  8.799  +  0.402BL  +  0.166ER  +  0.1 38DR 

+  5.611SD  -  1.148BL2  -  0.945ER2  -0.851DR2 
+  0.226SD2  -  0.4S0BL*ER  -  0.339BL*DR 
+  0.542BL*SD  -  0.636ER*DR  +  0.261ER*SD 
-  0.003DR*5D 


Table  B-13 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  CHER 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

E 

Regression 

(14) 

226.14 

1.86 

0.0297 

Buffer  Length  (BL) 

1 

15.63 

1.80 

0.1810 

Echo  Rate  (ER) 

1 

101.42 

11.65 

0.0007 

Display  Rate  (DR) 

1 

9.75 

1.12 

0.2905 

System  Delay  (SD) 

1 

1.25 

0.14 

0.7049 

BL2 

1 

22.95 

2.64 

0.1053 

ER2 

1 

32.19 

3.70 

0.0552 

DR2 

1 

15.25 

1.75 

0.1864 

SD2 

1 

4.43 

0.51 

0.4759 

BL*ER 

1 

3.75 

0.43 

0.5118 

BL*DR 

1 

2.06 

0.24 

0.6264 

BL*SD 

l 

0.66 

0.08 

0.7832 

ER*DR 

1 

5.94 

0.68 

0.4093 

ER*SD 

1 

10.16 

1.17 

0.2807 

DR*SD 

1 

0.66 

0.08 

0.7832 

Residual 

385 

3351.75 

Total 

399 

3577.89 

Second-order  Polynomial  Regression 


CHER  =  2.814  -  0.221BL  +  0.562ER  +  0.174DR 

+  0.062SD  -  0.423BL2  -  0.50 1ER2  -  0.345DR2 
+  0.186SD2  +  0.121BL*ER  +  0.089BL*DR 
+  0.050BL*SD  -  0.152ER*DR  +  0.199ER*SD 
-  0.050DR*SD 


R2  =  .063 


B-13 


Table  B-14 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  CKT 


ANOVA  Summary  Table 


Source  df 


Regression  (14) 

Buffer  Length  (BL)  1 

Echo  Rate  (ER)  1 

Display  Rate  (DR)  1 

System  Delay  (SD)  1 

BL2  1 

ER2  1 

DR2  1 

SD2  1 

BL*ER  1 

BL*DR  1 

BL*SD  1 

ER*DR  1 

ER*SD  1 

DR*SD  l 

Residual  385 


Total  399 


ss 

F 

R 

2876003.23 

1.97 

0.0188 

100836.95 

0.97 

0.3257 

144235.99 

1.39 

0.2399 

1177252.43 

11.31 

71005.53 

0.68 

0.4094 

19272.11 

0.18 

0.6676 

79005.14 

0.76 

0.3844 

26.58 

0.9871 

54919.20 

0.53 

0.4681 

195750.94 

1.88 

0.1711 

114793.91 

1.10 

0.2944 

506143.31 

4.86 

0.0281 

16528.31 

0.16 

0.6905 

65248.31 

0.63 

0.4291 

330984.47 

40088219.87 

3.18 

0.0754 

42964223.11 


Second-order  Polynomial  Regression 


CKT  =  152.69  -  17.75BL  +  21.23ER  -  60.65DR 

-  14.89SD  +  12.26BL2  +  24.84ER2  -  0.46DR2 
f  20.71  SD2  -  27.65BL*ER  +  21.17BL*DR 
+  44.46BL*SD  -  8.03ER*DR  -  15.96ER*SD 
+  35.95DR*SD 


R2  =  .066 


Table  B-15 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  TONR 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

P 

Regression 

(14) 

260.86 

3.58 

0.0001 

Buffer  Length  (BL) 

1 

0.07 

0.01 

0.9032 

Echo  Rate  (ER) 

1 

118.37 

22.73 

0.0001 

Display  Rate  (DR) 

1 

22.05 

4.23 

0.0403 

System  Delay  (SD) 

1 

8.76 

1.68 

0.1953 

BL2 

l 

4.20 

0.81 

0.3696 

ER2 

1 

17.40 

3.34 

0.0684 

DR2 

1 

1.80 

0.35 

0.5566 

SD2 

1 

4.80 

0.92 

0.3372 

BL*ER 

1 

10.56 

2.03 

0.1553 

BL*DR 

1 

10.56 

2.03 

0.1553 

BL*SD 

1 

33.60 

6.35 

0.0122 

ER*DR 

1 

5.06 

0.30 

0.5842 

ER*SD 

1 

1.56 

0.97 

0.3248 

DR*SD 

1 

22.56 

4.33 

0.0381 

Residual 

385 

2005.49 

Total 

399 

2266.36 

Second-order  Polynomial  Regression 


TONR  =  5.910  +  0.015BL  -  0.608ER  +  0.262DR 

-  0.165SD  +  0.181BL2  +  0.368ER2  +  0.118DR2 

-  0. 193SD2  -  0.203BL*ER  -  0.203BL*DR 

-  0.359BL*SD  -  0.078ER*DR  ♦  0.140ER*SD 

-  0.296DR*SD 


R2  =  .115 


Table  B-16 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  SDR 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

2 

Regression 

(14) 

789.52 

30.95 

0.0001 

Buffer  Length  (BL) 

1 

0.09 

0.05 

0.8149 

Echo  Rate  (ER) 

l 

20.00 

10.98 

0.0010 

Display  Rate  (DR) 

1 

1.33 

0.73 

0.3921 

System  Delay  (SD) 

1 

693.83 

380.81 

0.0001 

BL2 

1 

4.50 

2.47 

0.1170 

ER2 

1 

8.00 

4.39 

0.0368 

DR2 

1 

8.00 

4.39 

0.0368 

SD2 

1 

17.99 

9.88 

0.0018 

BL*ER 

1 

4.00 

2.20 

0.1392 

BL*DR 

1 

1.00 

0.55 

0.4592 

BL*SD 

1 

2.25 

1.23 

0.2671 

ER*DR 

1 

16.00 

8.78 

0.0032 

ER*SD 

1 

6.25 

3.43 

0.0648 

DR*SD 

1 

6.25 

3.43 

0.0648 

Residual 

385 

701.47 

Total 

399 

1491.00 

Second-order  Polynomial  Regression 


5DR=  2.100  +  0.01 7BL  -  0.250ER  +  0.064DR 

-  1.472SD  +  0.187BL2  +  0.249ER2  +  0.249DR2 
+  0.375SD2  -  0.125BL*ER  -  0.062BL*DR 

+  0.093BL*SD  +  0.250ER*DR  +  0.156ER*SD 

-  0.1 56DR*SD 


R2  =  .329 


B-16 


Table  B-17 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  DSPR 


ANOVA  Summary  Table 


Source 

Regression 

Buffer  Length  (BL) 

Echo  Rate  (ER) 

Display  Rate  (DR) 

System  Delay  (SD) 

BL2 

ER2 

DR2 

SD2 

BL*ER 

BL*DR 

BL*SD 

ER*DR 

ER*SD 

DR*SD 

Residual 

Total 


df 

SS 

(14) 

249.96 

1 

1.28 

1 

3.89 

1 

26.05 

1 

28.22 

1 

49.98 

1 

84.51 

1 

0.49 

1 

1.99 

1 

12.25 

1 

36.00 

1 

0.00 

1 

1.00 

1 

4.00 

1 

0.25 

385 

1514.03 

399 

1764.00 

F 

2 

4.54 

0.0001 

0.33 

0.5679 

0.99 

0.3201 

6.63 

0.0104 

7.18 

0.0077 

12.72 

0.0004 

21.49 

0.0001 

0.13 

0.7219 

0.51 

0.4764 

3.12 

0.0784 

9.15 

0.0026 

0.00 

1.0000 

0.25 

0.6144 

1.02 

0.3138 

0.06 

0.8011 

Second-order  Polynomial  Regression 


DSPR  =  4.200  +  0.063BL  -  0.1 101  ER  -  0.285DR 

-  0.296SD  -  0.625BL2  +  0.81 2ER2  +  0.062DR2 
+  0.124SD2  +  0.218BL*ER  -  0.375BL*DR 
+  0.000BL*SD  -  0.062ER*DR  +  0.125ER*SD 
+  0.031  DR*SD 


Table  B-18 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  ERR 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

Regression 

(14) 

732.03 

28.69 

0.0001 

Buff*  r  Length  (BL) 

1 

4.66 

2.56 

0.1106 

Echo  Rate  (ER) 

1 

6543.20 

358.93 

0.0001 

Display  Rate  (DR) 

1 

4.66 

2.56 

0.1106 

System  Delay  (SD) 

1 

12.63 

6.93 

0.0088 

BL2 

1 

0.08 

0.04 

0.8349 

ER2 

1 

14.58 

8.00 

0.0049 

DR2 

1 

0.08 

0.04 

0.8349 

SD4 

1 

27.37 

15.02 

0.0001 

BL*ER 

1 

1.00 

0.55 

0.4593 

BL*DR 

1 

9.00 

4.94 

0.0269 

BL*SD 

1 

0.25 

0.14 

0.7113 

ER*DR 

1 

1.00 

0.55 

0.4593 

ER#SD 

1 

2.25 

1.23 

0.2672 

DR*SD 

1 

0.25 

0.14 

0.7113 

Residual 

385 

701.72 

Total 

399 

1433.76 

Second-order  Polynomial  Regression 


ERR  =  2.660  -  0.120BL  -  1.429ER  -  0.120DR 

-  0.198SD  +  0.024BL2  ♦  0.337ER2  ♦  0.024DR2 
♦  0.462SD2  -  0.062BL*ER  -  0.187BL*DR 

-  0.031BL*SD  -  0.062ER*DR  -  0.093ER*SD 
+  0.031  DR*SD 


R2  =  .510 


Table  B-19 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  BLR 


ANOVA  Summary  Table 


Source 

Regression 

Buffer  Length  (BL) 

Echo  Rate  (ER) 

Display  Rate  (DR) 

System  Delay  (SD) 

BL2 

ER2 

DR2 

SD2 

BL*ER 

BL*DR 

BL*SD 

ER*DR 

ER*SD 

DR*SD 

Residual 

Total 


df 

SS 

(14) 

640.64 

1 

98.94 

1 

298.87 

1 

0.03 

1 

26.61 

1 

25.90 

1 

58.34 

1 

3.92 

1 

48.01 

1 

72.25 

1 

0.25 

1 

1.00 

1 

0.00 

1 

6.25 

1 

0.25 

385 

766.39 

399 

1407.04 

F 

£ 

22.99 

0.0001 

49.71 

0.0001 

150.14 

0.0001 

0.02 

0.8957 

13.37 

0.0003 

13.03 

0.0003 

29.29 

0.0001 

1.97 

0.1616 

24.12 

0.0001 

36.30 

0.0001 

0.13 

0.7232 

0.50 

0.4789 

0.00 

1.0000 

3.14 

0.0772 

0.13 

0.7232 

Second-order  Polynomial  Regression 


BLR  =  2.870  +  0.556BL  -  0.966ER  +  0.101  DR 

-  0.288SD  -  0.450BL2  +  0.675ER2  +  0.174DR2 
+  0.612SD2  ♦  0.531BL*ER  -  0.031BL*DR 

-  0.062BL*SD  +  0.000ER*DR  -  0.156ER*SD 

-  0.031  DR*SD 


R2  =  .455 


Table  B-20 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  SPEED 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

R 

Regression 

(14) 

2016.20 

38.89 

Buffer  Length  (BL) 

1 

20.98 

5.67 

0.0178 

Echo  Rate  (ER) 

1 

1038.87 

280.57 

0.0001 

Display  Rate  (DR) 

l 

8.99 

2.43 

0.1199 

System  Delay  (SD) 

1 

486.36 

131.35 

0.0001 

BL2 

1 

0.18 

0.05 

0.8268 

ER2 

1 

106.62 

28.77 

0.0001 

DR2 

1 

15.68 

4.23 

0.0404 

SD2 

1 

137.74 

37.20 

0.0001 

BL*ER 

1 

4.00 

1.08 

0.2993 

BL*DR 

1 

12.25 

3.31 

0.0697 

BL*SD 

1 

6.25 

1.69 

0.1947 

ER*DR 

1 

9.00 

2.43 

0.1198 

ER*SD 

l 

169.00 

45.64 

DR*SD 

1 

0.25 

0.07 

Residual 

385 

1425.55 

Total 

399 

3441.76 

Second-order  Polynomial  Regression 


SPEED  =  1.790  +  0.256BL  -  1.801  ER  -  0.167DR 

-  1.232SD  +  0.037BL2  +  0.912ER2  ♦  0.349DR2 
+  1.037SD2  +  0.125BL*ER  -  0.218BL*DR 

-  0.156BL*SD  +  0.187ER*DR  +  0.812ER*SD 
+  0.031  DR*SD 


R2  = .385 


B-20 


Table  B-21 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  ACCUR 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

P 

Regression 

(14) 

1579.03 

23.78 

0.0001 

Buffer  Length  (BL) 

1 

14.86 

3.13 

0.0775 

Echo  Rate  (ER) 

1 

935.71 

197.27 

0.0775 

Display  Rate  (DR) 

1 

4.36 

0.92 

0.3380 

System  Delay  (SD) 

1 

232.64 

49.05 

0.0001 

BL2 

1 

0.04 

0.01 

0.9234 

ER2 

1 

108.07 

22.77 

0.0001 

DR2 

1 

3.64 

0.77 

0.3819 

SD2 

1 

93.81 

19.78 

•  0.0001 

BL*ER 

l 

0.06 

0.01 

0.9087 

BL*DR 

1 

18.06 

3.81 

0.0517 

BL*SD 

1 

27.56 

5.81 

0.0164 

ER*DR 

27.56 

5.81 

0.0164 

ER*SD 

1 

105.06 

22.15 

0.0001 

DR*SD 

1 

7.56 

1.59 

0.2075 

Residual 

385 

1826.20 

Total 

399 

3405.24 

Second-order  Polynomial  Regression 


ACCUR  =  2.830  +  0.215BL  -  1.710ER  -  0.1 16DR 

-  0.852SD  ♦  0.019BL2  ♦  0.918ER*  +  0.168DR2 
♦  0.856SD2  ♦  0.015BL*ER  -  0.265BL*DR 

-  0.328BL*SD  +  0.328ER*DR  +  0.640ER*SD 
+  0.171DR*SD 


R*  =  .463 


Table  B-22 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  OVER 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

E 

Regression 

(14) 

1734.38 

29.67 

0.0001 

Buffer  Length  (BL) 

1 

11.61 

2.78 

0.0961 

Echo  Rate  (ER) 

1 

1007.80 

241.39 

0.0001 

Display  Rate  (DR) 

1 

29.80 

7.14 

0.0079 

System  Delay  (SD) 

1 

269.26 

64.49 

0.0001 

BL2 

1 

0.00 

0.00 

0.9736 

ER2 

1 

146.24 

35.01 

0.0001 

DR2 

1 

8.40 

2.01 

0.1571 

SD2 

1 

99.36 

23.80 

0.0001 

BL*ER 

1 

3.06 

0.73 

0.3923 

BL*DR 

1 

3.06 

0.73 

0.3923 

BL*SD 

1 

22.56 

5.40 

0.0206 

ER*DR 

1 

27.56 

6.60 

0.0106 

ER*SD 

1 

105.06 

25.16 

0.0001 

DR*SD 

1 

0.56 

0.13 

0.7138 

Residual 

00 

rr 

1607.37 

Total 

399 

3341.76 

Second-order  Polynomial  Regression 


OVER  =  2.390  +  0.190BL  -  1.774ER  -  0.305DR 

-  0.91 7SD  +  0.005BL2  +  1.068ER2  +  0.256DR2 
+  0.881SD2  +  0.109BL*ER  -  0.109BL*DR 

-  0.029BL*SD  +  0.328ER*DR  +  0.640ER*SD 
+  0.046DR*SD 


Table  C-l 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  PRODUCTION 


PRODUCTION 


Second-order  Polynomial  Regression 


-  0.4313  +  0.0630 BL  -  0.5747ER  -  0.0485DR 

-  0.6029SD  -  0.0221  BL2  +  0.2973ER2  +  0.0709DR2 
+  Q.1930SD2  +  Q.0551BL*ER  -  0.0641BL*DR 

-  0.0664BL*SD  +  0.0463ER*DR  +  0.1301ER*SD 

-  0.0175DR*SD 

R2  =  .623 


ANOVA  Summary  Table 


Source 

df 

SS 

F 

E 

Regression 

(14) 

206.62 

29.54 

0.0001 

Buffer  Length  (BL) 

1 

5.38 

10.79 

0.0011 

Echo  Rate  (ER) 

1 

91.66 

183.45 

0.0001 

Display  Rate  (DR) 

1 

2.21 

4.44 

0.0358 

System  Delay  (SD) 

1 

98.02 

196.17 

0.0001 

BL2 

1 

0.76 

1.53 

0.2169 

ER2 

1 

2.25 

4.52 

0.0342 

DR2 

1 

0.39 

0.79 

0.3739 

SD2 

1 

0.71 

1.43 

0.2323 

BL*ER 

1 

0.37 

0.76 

0.3848 

BL*DR 

1 

1.95 

3.91 

0.0488 

BL*SD 

1 

0.94 

1.88 

0.1706 

ER*DR 

1 

1.40 

2.81 

0.0944 

ER*SD 

1 

0.12 

0.25 

0.6193 

DR*SD 

1 

0.39 

0.78 

0.3770 

Residual 

385 

192.37 

Total 

399 

399.00 

Second-order  Polynomial  Regression 


WAITING  =  -  0.1487  ♦  0.1297BL  -  0.5352ER  +  0.832DR 

+  0.5534SD  -  0.0773BL2  +  0.1328ER2  +  0.0556DR2 
+  0.0747SD2  +  0.0384BL*ER  -  0.0873BL*DR 

-  0.0606BL*SD  -  0.0740ER*DR  +  0.0219ER*SD 

-  0.0390DR*SD 


R2  =  .517 


Table  C-3 


Analysis  of  Variance  Summary  Table  of  Second-order, 
Polynomial  Regression  of  PLANNING 


ANOVA  Summary  Table 


Source 

Regression 
Buffer  Length  (BL) 
Echo  Rate  (ER) 
Display  Rate  (DR) 
System  Delay  (SD) 
BL2 
ER2 
DR2 
SD2 
BL*ER 
BL*DR 
BL*SD 
ER*DR 
ER*SD 
DR*SD 

Residual 


Total 


df 

SS 

(14) 

64.92 

1 

0.26 

1 

17.10 

1 

1.58 

1 

22.46 

1 

0.16 

1 

1.36 

1 

0.13 

1 

16.78 

1 

0.05 

1 

0.03 

1 

0.72 

1 

0.75 

1 

0.29 

1 

3.17 

385 

334.07 

399 

399.00 

F 

E 

5.34 

0.0001 

0.31 

0.5786 

19.71 

0.0001 

1.83 

0.1769 

25.89 

0.0001 

0.19 

0.6622 

1.57 

0.2113 

0.15 

0.6945 

19.34 

0.0001 

0.06 

0.8003 

0.04 

0.8329 

0.84 

0.3597 

0.87 

0.3502 

0.34 

0.5577 

3.66 

0.0566 

Second-order  Polynomial  Regression 


PLANNING  =  -  0.4268  +  0.028BL  -  0.231  ER  -  0.070DR 

+  0.264SD  +  0.036BL2  +  0.103ER2  +  0.032DR2 
+  0.362SD2  ♦  0.014BL*ER  +  0.0I2BL*DR 
+  0.053BL*SD  +  0.054ER*DR  -  0.034ER*SD 
+  0.111DR*SD 
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